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Abstract 

Since the passage of the No Child Left Behind Act in 2002, underperforming schools have had to 
adopt research-based models for school reform, without a practical understanding of what that means. 
This article brings more clarity for educational leaders as they come to decide on how to fit a reform 
model to their school-specific contexts. 



NOTE: This module has been peer-reviewed, accepted, and sanctioned by the National Council of 
Professors of Educational Administration (NCPEA) as a scholarly contribution to the knowledge 
base in educational administration. 


1 Introduction 

Spurred by the No Child Left Behind Act (NCLB), the education community has increasingly turned to 
research to decide how best to improve schools. Such reliance on research-based approaches helps meet 
educational leaders’ and policymakers’ urgency to engage in efforts that will improve the lives of children. 

For school and district officials, however, it is not always clear how best to incorporate research-based 
approaches into school improvement plans. One obstacle is determining fit. Until recently, some school and 
district officials tended to seek programs that matched their own philosophy, paying less attention to how a 
program addressed school needs or affected student outcomes (Corcoran, 2003). Another is sorting through 
the research underlying each program. Even when educators and decision makers commit to adopting reform 
strategies with track records of effectiveness, they are challenged to find, interpret, and apply the relevant 
research (AIR, 2005). 
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Clearly, successful implementation of a reform program and positive student outcomes require both a 
philosophical and practical fit as well as a scientific base for the program. But how can school and district 
officials sort through the research and select a program that is scientifically based and right for their school? 

This article tackles this question through the lens of schoolwide reform by exploring the challenges 
schools and districts face in integrating research-based approaches into comprehensive improvement plans. 
In particular, this article offers suggestions on how schools and districts can become better consumers of 
school reform programs, emphasizing the need for (1) finding the right match between the reform approach 
and the school, (2) selecting an approach that has an adequate research base, and (3) considering costs—in 
terms of money, human resources, and time—associated with an approach. 

2 Selecting a Research-Based Approach 

A core element of whole school reform is the comprehensive improvement plan, and an important feature 
of any comprehensive improvement plans is the integration of research-based strategies and initiatives. For 
some schools and districts, however, selecting a reform strategy and integrating it into a comprehensive 
improvement plan can be a daunting task. Hundreds of reform programs claim to be rooted in scientific 
research and promise to turn around low-performing schools. Yet, only a small portion of reform programs 
meet NCLB’s stringent criteria for being scientifically based or have clearly established, positive effects on 
student achievement (Borman, Hewes, Overman, & Brown, 2003). School and district officials are challenged 
to find, interpret, and apply the relevant research on different programs. They also face practical constraints, 
such as costs, and pressure for immediate results. 

Rather than selecting the first approach presented to them or one that has been adopted by thousands 
of other schools, school and district officials need to approach the choice of a research-based reform program 
as an important and complicated consumer decision (Stringfield, 1998). As informed consumers, they must 
carefully consider (1) the fit between the reform approach and the school, (2) the rigor and effectiveness of 
the research, and (3) the resources and time required to properly implement the approach. 

2.1 Ensuring Fit 

School and district officials can face challenges when determining fit between a school and a particular reform 
program. For example, some schools may be tempted to select an approach that closely resembles programs 
already employed by the school or an approach that plays to the school’s strengths. Although this may make 
implementation easier to achieve, the resulting change in the school may be minimal or not substantive 
enough. Other schools may seek more radical change by selecting a reform program that strictly addresses 
the school’s diagnosed needs but maintains little coherence with its existing programs or philosophies. In 
such cases, the program can fail to take hold among school staff and implementation can stagger. 

To avoid these problems, schools and districts should carefully match the reform program to the needs, 
goals, and capacity of the school. This requires both a thorough needs assessment and a greater understanding 
of the types of reform programs available. 

2.1.1 What role does a needs assessment play? 

Being an informed consumer requires an understanding of a district’s or school’s capacity, needs, and goals 
and, more important, an understanding of the barriers to meeting these needs and attaining these goals 
(Fashola, 2004). Thorough needs assessments can provide both of these by looking at student demographics 
and achievement; classroom processes and supports such as curriculum and instruction, assessment, profes¬ 
sional development, and classroom management; and school management and resources (Hale, 2000). 

Under the Schoolwide Title I Program, schools in the planning process are required to conduct a compre¬ 
hensive needs assessment. The needs assessment guides the development of the comprehensive improvement 
plan and suggests benchmarks for its evaluation. As a result, the needs assessment is closely linked to all 
aspects of schoolwide program implementation (U.S. Department of Education, 2006). The non-regulatory 
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guidance for the Schoolwide Title I Program suggests school and district staff take the following steps in 
conducting a needs assessment: 

• Establish schoolwide planning teams: 

• Clarify the vision of reform; 

• Create a school profile based on student needs, curriculum and instruction, professional development, 
family and community involvement, school context, and organization 

• Identify data sources; and 

• Analyze data. 

Needs assessments allow school and district officials to focus on what is most important at school sites. 
They also help them to identify whether staff, administration, and students at a particular school have the 
motivation and capacity to implement a given reform. 

2.1.2 What types of reform approaches are available? 

Schools and districts have a variety of reform programs to consider for integration into a school’s comprehen¬ 
sive improvement plan. Some approaches provide their own curricula and instructional methods while others 
take schools through a process of decision-making centered on teaching and learning. Perhaps the best way 
to think about reform programs is along a continuum: from approaches focused primarily on curriculum and 
instruction to those focused primarily on processes and school organization (Hale, 2000). 

Curricular or instructional programs are designed to increase student academic achievement in specific 
curricular areas. They typically provide level- and grade-specific curriculum and testing materials, training 
sessions and manuals, and many other forms of technical assistance. Because of their academic focus, these 
programs tend to be more structured than the broader organizational programs. They most often cover basic 
curriculum areas such as reading and math, but may also address other subjects. To address their academic 
needs, schools often implement two or more different curricular programs concurrently (Fashola, 2004). 

Process or organizational programs focus on the organizational and administrative needs of the school 
rather than directly address academic achievement of specified subject areas. Instead of providing student 
materials and specific instructional approaches as curricular models often do, these programs restructure 
the organizational makeup of the school and improve the school climate through goal setting and similar 
activities. Process or organizational reform programs are rooted in the finding that schools are sometimes 
able to improve the academic achievement their students by restructuring the school organization (Fashola, 
2004). 

As schools and districts explore different reform programs, they should think about how a particular 
approach fits into the school’s overall vision and how it might be integrated with other strategies to form a 
coherent and comprehensive improvement plan that will help the school meet its needs and objectives. 

2.2 Sorting Through the Research 

NCLB legislation makes more than 100 references to scientifically based research and defines it as rigorous, 
systematic, objective, empirical, peer-reviewed research that relies on multiple measurements and observa¬ 
tions (U.S. Department of Education, 2006). Although such emphasis places greater responsibility on states 
and districts to demonstrate that the programs supported by federal funds are grounded in strong research, 
school and district officials are saddled with understanding, interpreting, and applying the research that 
exists. 

For schools that are just starting to implement schoolwide reform, there are a number of synthesis reports 
that chronicle the most widely implemented reform models and the scientifically based criteria by which their 
effectiveness is evaluated. More specifically, these reports apply standards of effectiveness to various program 
types according to rigor of research design, the agreement of outcomes across studies, and research-based 
program components (according to Title I Schoolwide Programs). Among the many possible resources, 
there are four reports in particular, which comprehensively review some of the most widely implemented 
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programs: the CSRQ Center Report on Elementary and Secondary School Comprehensive School Reform 
Models from the American Institutes for Research (AIR); the Catalog of School Reform Models compiled 
by the Northwest Regional Educational Laboratory (NWREL); Better by Design, a review of major school 
reform models produced by the Thomas B. Fordham Foundation; and the Comprehensive School Reform 
and Student Achievement: A Meta-Analysis published through the Center for Research on the Education 
of Students Placed At Risk (CRESPAR). These reports, taken together, address some basic questions that 
consumers should consider when adopting a school reform approach. 

2.2.1 What qualifies as methodological rigor? 

In recent years, rigor of research design methodology has referred to two designs in particular: experimental 
and quasi-experimental designs. Thought to be the more rigorous of the two, experimental designs set 
conditions in which students are randomly assigned to comparison groups (minimally one that receives the 
treatment or program and one that does not). Students in each group are tested on a valid and reliable 
measure before and after exposure to the treatment/ program. Quasi-experimental designs also assign 
students to control and treatment groups, but instead of being assigned randomly, students are matched 
according to a number of relevant student characteristics. This design allows for analysis of historical data 
and retrospective analysis while also statistically isolating the effects of the treatment/program on outcomes 
(Slavin & Fashola, 2003). 

2.2.2 What qualifies as evidence of effectiveness? 

Straightforward descriptive analyses may show differences in student achievement like that of gain scores 
or the mean change in scores from one year to another, however, what is not so clear is whether there are 
true differences in the scores, or whether they are the result of chance (Slavin & Fashola, 1998). Before 
beginning analysis, researchers typically set levels of significance (symbolized by p) at p <= .05 or p<= .01, 
which signifies that there is a five- or one-percent probability, respectively, that the differences are a result 
of chance. Establishing significance, however, only tells us with great certainty that the differences between 
two sets of mean scores are not the result of chance; they say nothing of the extent to which the means are 
different—for example between pre and post test scores comparing the performance between the control and 
treatment groups. The effect size has been described as “a standard means of expressing achievement gains 
and losses across studies, showing differences between experimental and control groups in terms of standard 
deviation” Slavin, & Fashola 1998). In practice, +0.20 to +0.30 is considered a small effect, +0.50 a medium 
effect, and +0.80 a large effect (Cohen, 1988). While programs evaluated with experimental designs and 
with strong evidence of improving student outcomes are considered the gold standard in educational research, 
few programs meet the rigorous standards set by NCLB. Schools may need to consider programs without 
evidence of effectiveness that have individual components with strong links to proven research. 

2.2.3 What other factors should be considered? 

While scientifically-based criteria for rigorous research is indeed helpful in guiding schools’ choices of the 
programs they will adopt, there are still newer programs that have not yet been evaluated or have not 
been evaluated enough. Slavin has called such programs “promising, but not proven” (Slavin & Fashola, 
1998). Besides being new, other programs that may fit this category are those that were found to have a 
positive influence on student outcomes in research. These outcomes may either be qualitative in nature or 
quantitative studies that did not use an experimental or quasi-experimental design. 

Although rigor of research design is important, consumers should be careful not to rely solely on statistical 
genera]izability when deciding on a match with a particular program. Even if the research supporting an 
a reform program meets all the criteria of methodological rigor, shows positive effects, results in moderate 
to strong effects sizes, and seems to perfectly match the needs identified in the comprehensive assessment, 
it is still necessary to study the contexts in which the program was implemented. In other words, factors 
such as school culture, support internal and external to the school, or conditions of school facilities and 
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resources typically do not, show up in quantitatively driven evaluations. Rather than the decisive factor, 
generalizability should serve as further evidence when considering a model. 

2.3 Practical Considerations 

In addition to ensuring proper fit and assessing the research base, school and district officials need to consider 
whether a given reform program is practical for their school. This means taking into account the resources— 
both human and financial—and the time necessary to implement the program. 

2.3.1 Do we have the human resources and capacity to support the reform program? 

All teachers and administrators, especially ones at low-performing schools, need a coherent system of profes¬ 
sional development. However, many districts lack support systems to provide teachers and principals with 
the training they need. Most research-based programs require training prior to and during implementation. 
These trainings help to focus professional development across grades and subjects and should be the center 
of the professional development component of the school’s comprehensive improvement plan. Every effort 
should be made to ensure that other forms of professional development complement these trainings in terms 
of content, philosophy, time required, and costs. To do otherwise will result in a fragmented system of 
professional development and will be less effective in changing classroom practices. 

Closely related to the need for professional development is the need for external assistance. Research has 
shown that program effectiveness is largely dependent on implementation support (Newmann, & Sconzert 
2000). Effective implementation assistance should involve a blend of initial training and ongoing, on-site 
follow-up support for all teachers. It is essential that this support be provided by highly qualified model 
staff or by a local trainer who themselves has received sufficient training to be considered an expert in the 
research based approach. Capacity building is also vital, as it helps schools develop local expertise and 
establish structures that nurture continued learning (Northwest Regional Educational Laboratory & The 
National Clearinghouse for Comprehensive School Reform, 2003). 

According to a recent American Institutes for Research report (American Institutes for Research. 2005), 
questions schools and districts should ask before selecting a research-based approach include: 

• Does the provider track and support full implementation in schools? 

• Does the provider offer comprehensive training opportunities and supporting materials? 

• Does the provider ensure that professional development effectively supports full program implementa¬ 
tion? 

• Does the provider develop a school’s internal capacity to provide professional development? 


2.3.2 Do we have the financial resources needed to adopt the reform program? 

School staffs have to make hard choices about their budgets. Start-up costs for implementing a research- 
based program can carry a hefty price tag. There are costs associated with materials, trainings, and teacher 
release time. Some may choose a particular program largely because it is inexpensive. But choosing a 
program based on costs without regard for fit or rigor is unlikely to produce effective schools or may even 
be harmful. 

Schoolwide Title I program schools have the flexibility to coordinate and consolidate resources and services 
to improve the achievement of all students in a school. One key benefit of operating a Schoolwide Title I 
program school is the ability to integrate federal, state, and local education funds. Federal discretionary 
grants and formula grants, with a few exceptions, can be consolidated and used to raise student achievement 
across an entire school. Moreover, many statutory and regulatory reporting requirements are waived for 
Title I schoolwide program schools, however schools must demonstrate that the intent and purposes of the 
individual federal programs are met. For example, a school operating a schoolwide program may demonstrate 
that it meets the intent and purpose of the Title II, Part A program (Preparing, Training, and Recruiting 
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High Quality Teachers and Principals) if a school’s comprehensive improvement plan identifies activities that 
increase teacher quality by providing professional development trainings associated with a research based 
program. 

Whenever possible, school officials should consider integrating funds to finance the following costs asso¬ 
ciated with adopting research-based programs: 

• Materials (training materials, textbooks, assessments, technology) 

• Training (in-service workshops, train-the-trainer sessions, leadership training, data analysis, follow-up 
support) 

• Staff (on-site coaches, facilitators, lead teachers, release time for teachers, stipends, substitute teacher 
costs). 

Since the amount and quality of human resources is dependent on the availability of financial resources, school 
officials operating schoolwide programs should carefully consider opportunities to integrate costs associated 
with adopting a research-based approach. 

2.3.3 How much time will the reform program take to produce results (and do we have that 
much time)? 

Schools and school districts feel pressure from parents, community stakeholders, and local and state ac¬ 
countability systems to act quickly and produce results. In today’s high stakes accountability environment, 
parents and the general public pressure schools to raise test scores. Superintendents feel pressure to hastily 
implement new programs to show that they are doing “something” during their tenure. Moreover, in the 
era of NCLB, low-performing schools are pressured to produce gains in the narrow timeline from initial 
identification as a school in need of improvement to school restructuring. 

Often, districts respond to this pressure by mandating reforms before evidence of their effectiveness has 
been proven. Research demonstrates that reforms require 3 to 5 years to become fully implemented and to 
show improvement in student performance. Yet district leaders feel that they can’t wait years to produce 
gains in student outcomes (Corcoran, 2003). The need for expediency rather than proven results drives 
decision-making. 

The need for scientifically based research becomes even more important in light of the pressure schools 
and districts face to act quickly to improve student achievement. Without readily available, rigorous research 
about education practices, educators and policymakers are more likely to make rushed decisions that are 
ineffective or even harmful (Lauer, 2004). 

3 Summary 

The integration of research-based approaches into comprehensive improvement plans has the potential to 
transform schools. For this to happen, however, school, district, and state officials must realize that pursuing 
educational reform means entering a marketplace of competing reform approaches. 

School and district officials must be savvy consumers. They must identify their school’s needs and seek 
a reform approach that will help their school achieve its goals. They must have a basic understanding of 
what the research says and be able to recognize a reform approach with a rigorous and reliable research 
base. They must be cognizant of practical issues such as human and financial resources, as well as the time 
necessary to fully implement a given approach. 

For today’s educational stakeholders, the efforts required to choose, implement, and institutionalize a 
school reform program are sizeable. However, the benefits to districts, schools, and—most importantly— 
students can be equally great. 
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